Reliability generalization: lapsus linguae

RELIABILITY GENERALIZATION: LAPSUS LINGUAEby JULIE M. SMITHDecember 2011Advisor: Dr. Shlomo S. SawilowskyMajor: Education Evaluation and ResearchDegree: Doctor of PhilosophyThis study examines the proposed Reliability Generalization (RG) method for studyingreliability. RG employs the application of meta-analytic techniques similar to those used invalidity generalization studies to examine reliability coefficients. This study explains why RGdoes not provide a proper research method for the study of reliability, including describing howreliability is not a singular metric but a family of coefficients that are not interchangeable, alongwith other issues, such as sample and test administration. This research used Monte Carlosimulations designed to illustrate how the same instrument, administered repeatedly, can result indifferent reliability coefficients and to show that variation in reliability coefficients is due tosampling error; results illustrate that the reliability of a test will vary across test administrations based on the size and composition of the sample and how the sample was selected (randomly versus non-randomly).